Word Segmentation needs change- From a linguist's view

Word Segmentation Based on Estimation of Words from Examples

From a cognitive point of view, words can be recognized based on learned data which can be obtained from linguistic materials. Namely, people learn words from many examples which they meet. We propose a word segmentation algorithm based on estimated knowledge for words acquired from both local texts being processed and POS tagged corpus. In order to show the feasibility of our model, we apply i...

Word segmentation in Persian continuous speech using F0 contour

Word segmentation in continuous speech is a complex cognitive process. Previous research on spoken word segmentation has revealed that in fixed-stress languages, listeners use acoustic cues to stress to de-segment speech into words. It has been further assumed that stress in non-final or non-initial position hinders the demarcative function of this prosodic factor. In Persian, stress is retract...

Sticking to the Text: a Corpus Linguist's View of Language

Keyboard Logs as Natural Annotations for Word Segmentation

In this paper we propose a framework to improve word segmentation accuracy using input method logs. An input method is software used to type sentences in languages which have far more characters than the number of keys on a keyboard. The main contributions of this paper are: 1) an input method server that proposes word candidates which are not included in the vocabulary, 2) a publicly usable in...

Word Segmentation of Vietnamese Texts: a Comparison of Approaches

We present in this paper a comparison between three segmentation systems for the Vietnamese language. Indeed, the majority of Vietnamese words is built by semantic composition from about 7,000 syllables, that also have a meaning as isolated words. So the identification of word boundaries in a text is not a simple task, and ambiguities often appear. Beyond the presentation of the tested systems,...